Order estimation and sequential universal data compression of a hidden Markov source by the method of mixtures

Abstmet-We consider first the estimation of the order, i.e., the number of states, of a discrete-time finite-alphabet stationary ergodic hidden Markov source (HMS). Our estimator uses a description of the observed data in terms of a uniquely deadable code with respect to a mixture distriiw obtained by suitably mixing a parametric family of dletribntiom on the observation space. This procedure avoids nrsxinvlm likelihood calculations. The order estimator is shown to be strongly eongistent with the probability of underestim;rtion decaying exponentialIy fast in the number n of observations, -e the prthbility of overestimation does not exceed cn -', where E is a constant.. Next, we present a seqoential algorithm fw the uniquely decodable univerapl data compression of the IIMS, which performs an on-line estimation of source order followed by arithmetic COQins. This code asymptotically attains optimum average redundancy.